Skip to contentSkip to conversation

tar: add ptest support

PATCH Superseded [email protected] 2 participants 4 messages Sep 25, 2023 – Sep 26, 2023

v5 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

PATCH v5 1/1 tar: add ptest support

From: Yan <[email protected]>

Add a ptest for tar.
- It is taking around 3m to execute with kvm, so added it to PTEST_SLOW.
- It contains 242 cases.
- Below is parts of the run log:
START: ptest-runner
2023-09-25T05:06
BEGIN: /usr/lib/tar/ptest
## ------------------------ ##
## GNU tar 1.35 test suite. ##
## ------------------------ ##
PASS: tar version
PASS: decompressing from stdin
...
## ------------- ##
## Test results. ##
## ------------- ##

198 tests were successful.
44 tests were skipped.
DURATION: 188
END: /usr/lib/tar/ptest
2023-09-25T05:09
STOP: ptest-runner
TOTAL: 1 FAIL: 0

---
 .../distro/include/ptest-packagelists.inc     |  1 +
 meta/recipes-core/images/core-image-ptest.bb  |  3 ++
 meta/recipes-extended/tar/tar/ptest.patch     | 23 +++++++++++++
 meta/recipes-extended/tar/tar/run-ptest       | 14 ++++++++
 meta/recipes-extended/tar/tar_1.35.bb         | 32 +++++++++++++++++++
 5 files changed, 73 insertions(+)
 create mode 100644 meta/recipes-extended/tar/tar/ptest.patch
 create mode 100644 meta/recipes-extended/tar/tar/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
index 9160103cb0..3df7c9e405 100644
@@ -109,6 +109,7 @@ PTESTS_SLOW = "\
     python3-cryptography \
     python3 \
     strace \
+    tar \
     tcl \
     util-linux \
     valgrind \
meta/recipes-core/images/core-image-ptest.bb +3 -0
index b81ab7b7c8..b6f5c2fd60 100644
@@ -24,6 +24,9 @@ IMAGE_ROOTFS_EXTRA_SPACE:virtclass-mcextend-mdadm = "1524288"
 IMAGE_ROOTFS_EXTRA_SPACE:virtclass-mcextend-strace = "1024288"
 IMAGE_ROOTFS_EXTRA_SPACE:virtclass-mcextend-lttng-tools = "1524288"
 
+# tar-ptest in particular needs more space
+IMAGE_ROOTFS_EXTRA_SPACE:virtclass-mcextend-tar = "1524288"
+
 # ptests need more memory than standard to avoid the OOM killer
 QB_MEM = "-m 1024"
 QB_MEM:virtclass-mcextend-lttng-tools = "-m 4096"
meta/recipes-extended/tar/tar/ptest.patch new file +23 -0
new file mode 100644
index 0000000000..4a5951211e
@@ -0,0 +1,23 @@
+tar: Fix bug of compiling testsuite
+
+Del exclude17.at and exclude18.at from testsuite.at,
+because these files are not exist in tarball.
+
+Upstream-Status: Inappropriate [oe specific]
+
+Signed-off-by: Qiu Tingting <[email protected]>
+
+diff -rNu a/tests/testsuite.at b/tests/testsuite.at
+--- a/tests/testsuite.at	2023-09-25 10:39:52.176684379 +0800
++++ b/tests/testsuite.at	2023-09-25 10:39:25.532684766 +0800
+@@ -312,8 +312,8 @@
+ m4_include([exclude14.at])
+ m4_include([exclude15.at])
+ m4_include([exclude16.at])
+-m4_include([exclude17.at])
+-m4_include([exclude18.at])
++# m4_include([exclude17.at])
++# m4_include([exclude18.at])
+ 
+ AT_BANNER([Deletions])
+ m4_include([delete01.at])
meta/recipes-extended/tar/tar/run-ptest new file +14 -0
new file mode 100644
index 0000000000..185b33d61a
@@ -0,0 +1,14 @@
+#!/bin/sh
+
+# Define tar test work dir
+WORKDIR=@PTEST_PATH@/tests/
+
+# Run test
+cd ${WORKDIR}
+./atconfig ./atlocal ./testsuite
+
+# clear log
+rm -rf testsuite.dir
+rm -rf testsuite.log
+
+./testsuite --am-fmt
meta/recipes-extended/tar/tar_1.35.bb +32 -0
index 4dbd418b60..9605ec52a6 100644
@@ -42,6 +42,38 @@ do_install:append:class-target() {
     fi
 }
 
+# add for ptest support
+SRC_URI += " \
+    file://run-ptest \
+    file://ptest.patch \
+"
+
+inherit ptest
+
+do_compile_ptest() {
+    oe_runmake -C ${B}/gnu/ check
+    oe_runmake -C ${B}/lib/ check
+    oe_runmake -C ${B}/rmt/ check
+    oe_runmake -C ${B}/src/ check
+    rm -rf ${S}/tests/testsuite
+    oe_runmake -C ${B}/tests/ testsuite
+    oe_runmake -C ${B}/tests/ genfile checkseekhole ckmtime
+}
+
+do_install_ptest() {
+    install -d ${D}${PTEST_PATH}/tests/
+    install --mode=755 ${B}/tests/atconfig ${D}${PTEST_PATH}/tests/
+    sed -i "/abs_/d" ${D}${PTEST_PATH}/tests/atconfig
+    echo "abs_builddir=${PTEST_PATH}/tests/" >> ${D}${PTEST_PATH}/tests/atconfig
+    install --mode=755 ${B}/tests/atlocal ${D}${PTEST_PATH}/tests/
+    sed -i "/PATH=/d" ${D}${PTEST_PATH}/tests/atlocal
+    install --mode=755 ${B}/tests/genfile ${D}${PTEST_PATH}/tests/
+    install --mode=755 ${B}/tests/checkseekhole ${D}${PTEST_PATH}/tests/
+    install --mode=755 ${B}/tests/ckmtime ${D}${PTEST_PATH}/tests/
+    install --mode=755 ${S}/tests/testsuite ${D}${PTEST_PATH}/tests/
+    sed -i "s#@PTEST_PATH@#${PTEST_PATH}#g" ${D}${PTEST_PATH}/run-ptest
+}
+
 PACKAGES =+ "${PN}-rmt"
 
 FILES:${PN}-rmt = "${sbindir}/rmt*"
Signature · 1 line
2.25.1
#1 lore raw

meta/recipes-extended/tar/tar/ptest.patch line 6

Quoted 6 lines · On Mon, 25 Sept 2023 at 07:35, [email protected] <[email protected]> wrote:
meta/recipes-extended/tar/tar/ptest.patch new file +6 -0
+tar: Fix bug of compiling testsuite
+
+Del exclude17.at and exclude18.at from testsuite.at,
+because these files are not exist in tarball.
+
+Upstream-Status: Inappropriate [oe specific]

Thanks for persevering with this, we're nearly there.

When adding 'Inappropriate' patches, it is important to check if this issue has been resolved in upstream git, or if there's a ticket for it (and add a ticket if needed).
In this case, there's a fix, so please add the link to this patch:
https://git.savannah.gnu.org/cgit/tar.git/commit/?id=39849e9d91f477d3fb839f93cd0815d0cb3273e9

Upstreams should be notified of the problems when we add 'Inappropriate' patches to work around them.

Alex

#2 lore raw

Hi,Alex

Quoted 4 lines

In this case, there's a fix, so please add the link to this patch:
https://git.savannah.gnu.org/cgit/tar.git/commit/?id=39849e9d91f477d3fb839f93cd0815d0cb3273e9

The bug: There is no exclude17.at or exclude18.at file in src dir, so testsuite can not be compiled.

| /bin/bash '/Yocto/oe-core-test/build/tmp-glibc/work/core2-64-oe-linux/tar/1.35/tar-1.35/build-aux/missing' autom4te --language=autotest -I ../../tar-1.35/tests testsuite.at -o ../../tar-1.35/tests/testsuite.tmp | m4:../../tar-1.35/tests/testsuite.at:315: cannot open exclude17.at': No such file or directory | m4:../../tar-1.35/tests/testsuite.at:316: cannot open exclude18.at': No such file or directory | autom4te: error: m4 failed with exit status: 1 Up patch only added exclude17.at and exclude18.at into Makefile, but could not fix none file problem.
So, I think it is not the fix patch.

The key is that used tarball(*1) does not contain exclude17.at(*3) and exclude18.at(*4).
But another tarball(*2) contains the two files.
I am not sure why the same version has diffrenent tarballs.
For *1, testsuite is exist in src without compiling, so there is no problem. Of course there is not any ticket or fix patch. In oe ptest, for adding --am-fmt, recompiling testsuite is neccessary, so the problem appeared.

*1: https://ftp.gnu.org/gnu/tar/tar-1.35.tar.bz2
*2: https://git.savannah.gnu.org/cgit/tar.git/snapshot/tar-1.35.tar.gz
*3: The following patch add exclude17.at.
https://git.savannah.gnu.org/cgit/tar.git/commit/?id=0f0722df45ec520d0dac7c9ad7e69165e9140931
*4: The following patch add exclude18.at. ( src/names.c is changed)
https://git.savannah.gnu.org/cgit/tar.git/commit/?id=4f814e0e4c673f86dc65a557f7e55f6b5efd1529

#3 lore raw

Hello,

contents of tarball at *1 is determined by EXTRA_DIST setting in
https://git.savannah.gnu.org/cgit/tar.git/tree/tests/Makefile.am?id=39849e9d91f477d3fb839f93cd0815d0cb3273e9#n20
which in turn refers to TESTSUITE_AT.

As of 1.35 release, TESTSUITE_AT was missing exclude17/18.at and the following later commit adds them, fixing the issue of missing files:
https://git.savannah.gnu.org/cgit/tar.git/commit/?id=39849e9d91f477d3fb839f93cd0815d0cb3273e9
Note that the link in the commit message refers specifically to that:
https://lists.gnu.org/r/bug-tar/2023-07/msg00002.html

Commits in *3 and *4 added the files, but forgot to update TESTSUITE_AT.

Tarball in *2 has those files, because it is not actually a release tarball, it is an archive that git automatically generates from the git source tree when a new tag is added. That tarball should not be used.

Rather than modify testsuite.at to exclude the two missing files, maybe it's better to add the two missing files to the source tree in that same patch instead? And please do refer to the commit that will fix the issue upstream when they issue a new tarball release.

Alex

Quoted 33 lines · On Tue, 26 Sept 2023 at 03:18, [email protected] <[email protected]> wrote:

Hi,Alex

In this case, there's a fix, so please add the link to this patch:
https://git.savannah.gnu.org/cgit/tar.git/commit/?id=39849e9d91f477d3fb839f93cd0815d0cb3273e9

The bug: There is no exclude17.at or exclude18.at file in src dir, so testsuite can not be compiled.

| /bin/bash '/Yocto/oe-core-test/build/tmp-glibc/work/core2-64-oe-linux/tar/1.35/tar-1.35/build-aux/missing' autom4te --language=autotest -I ../../tar-1.35/tests testsuite.at -o ../../tar-1.35/tests/testsuite.tmp | m4:../../tar-1.35/tests/testsuite.at:315: cannot open exclude17.at': No such file or directory | m4:../../tar-1.35/tests/testsuite.at:316: cannot open exclude18.at': No such file or directory | autom4te: error: m4 failed with exit status: 1 Up patch only added exclude17.at and exclude18.at into Makefile, but could not fix none file problem.
So, I think it is not the fix patch.

The key is that used tarball(*1) does not contain exclude17.at(*3) and exclude18.at(*4).
But another tarball(*2) contains the two files.
I am not sure why the same version has diffrenent tarballs.
For *1, testsuite is exist in src without compiling, so there is no problem. Of course there is not any ticket or fix patch. In oe ptest, for adding --am-fmt, recompiling testsuite is neccessary, so the problem appeared.

*1: https://ftp.gnu.org/gnu/tar/tar-1.35.tar.bz2
*2: https://git.savannah.gnu.org/cgit/tar.git/snapshot/tar-1.35.tar.gz
*3: The following patch add exclude17.at.
https://git.savannah.gnu.org/cgit/tar.git/commit/?id=0f0722df45ec520d0dac7c9ad7e69165e9140931
*4: The following patch add exclude18.at. (src/names.c is changed)
https://git.savannah.gnu.org/cgit/tar.git/commit/?id=4f814e0e4c673f86dc65a557f7e55f6b5efd1529

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#188213): https://lists.openembedded.org/g/openembedded-core/message/188213
Mute This Topic: https://lists.openembedded.org/mt/101569333/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-